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APPLICATION TO WEAK KAM THEORY 
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Abstract. We prove continuity of certain cost functions arising from optimal 
control of affine control systems. We give sharp sufficient conditions for this con- 
tinuity. As an application, we prove a version of weak KAM theorem and consider 
the Aubry-Mather problems corresponding to these systems. 



1. Introduction 

Integrability of Hamiltonian systems has been a subject of considerable interest 
for several decades. One way to understand the dynamics of such systems is to find 
a family of smooth solutions, called generating functions, to the time-independent 
Hamilton-Jacobi equation. These generating functions define symplectic transfor- 
mations which transform the given completely integrable Hamiltonian system to a 
much simpler one that are easily solvable. 

On the contrary, if the Hamiltonian system is not completely integrable, then it 
is natural to ask whether one can solve the Hamilton-Jacobi equation in certain 
weak sense. This is accomplished in, what is known as, the weak KAM theorem 
under certain assumptions on the Hamiltonian. More precisely, let L : TM — > R be 
a Lagrangian defined on the tangent bundle TM of a compact manifold M which 
satisfies the following conditions: 

(1) the restriction of the Lagrangian L to each tangent space has positive definite 
Hessian, 

(2) L(x,v) > C\v\ 2 + K for some Riemannian metric | • | and some constants 
K,C > 0. 

Let H : T*M — > R be the corresponding Hamiltonian defined by the Legendre 
transform: 

H(x,a) = sup [a{v ) — L(x, v )]. 

veT x M 

The following is the weak KAM theorem mentioned above. It was first proven in 
[12] when M is a torus and was extended to all compact manifolds in [8] (see also 
[TO] for a version related to vakonomic mechanics). 
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Theorem 1.1. Under the above assumptions, there exists a unique constant h such 
that the Hamilton- Jacobi equation 

(1.1) H(x,df x ) = -h, 
has a viscosity solution. 

In order to give the definition of viscosity solution, we first recall the concepts of 
sub- and super- differentials. If / is a continuous function on a manifold M, then 
the sub-differential d~ f x of the function / at a point x is the subset of the cotangent 
space T*M defined by the following: a co-vector p in the cotangent space T*M is 
contained in the sub-differential d~f x of / at x if there exists a smooth function g 
defined in a neighborhood O of x such that dg x = p and g touches / from above. By 
g touching from above, we mean that f(x) = g(x) and f(y) < g(y) for all y in the 
set O. The super-differential d + f of / is defined in a similar way with the function g 
touching from below instead. Let G : R x T*M -»Rbea continuous function, then 
a continuous function / is called a sub-solution to the equation G(f(x),x,p) = if 
for each p in the sub-differential d~ f x , 

G(f(x) : x,p)<0. 

Similarly, / is a super-solution if for each p in the super-differential d + f x , 

G(f(x),x,p)>0. 

If / is both a super and a sub-solution, then it is called a viscosity solution (see [5] 
for various different characterizations of the sub-differential and viscosity solution). 

In this paper, we study weak KAM theorem corresponding to Hamiltonians which 
arise from certain optimal control problems. More precisely, let Xq, X\, X n be 
smooth vector fields on a compact manifold M of dimension m and consider the 
following family of ODEs, called control-affine system: 

n 

(1.2) x(t)=F(x(t),u(t)) := X (x(t)) + J2 u >(t) X Mt)), 

i=i 

where u(-) := (ui(-), ...,u n (-)) : [0, T] — > R n are essentially bounded measurable 
functions, called controls, and solutions to (11.21) are Lipschitz curves in M, called 
admissible paths. 

Let L : MxR" ->Rbea smooth function, called Lagrangian. The optimal control 
cost ct corresponding to the above control affine system (11.21) and Lagrangian L is 
the following function: 

(1.3) c T (x, y) = inf I L(x(t),u(t))dt, 

Jo 

where the infimum is taken over all pairs (x(-), «(•)) which satisfies the affine control 
system (11.21) and the boundary conditions x(0) = x and x(T) = y. 

Since there may exist points which are not connected by any admissible path, 
the above cost function is not always well-defined without additional assumptions. 
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We recall that a family of vector fields {X\, X n } is said to be /c-generating if 
the vector fields and their iterated Lie brackets up to k — 1 order spanned each 
tangent space in TM. More precisely, the following holds for each point x in the 
manifold M 

T X M = span{[A n , [X i2 , [X^, Xj]](x) | 1 < i 5 < n, 1 < I < k}. 

The family {X\, ...,X n } is bracket generating if it is /c-generating for some k. If we 
assume that the family {X±, ...,X n } is bracket generating, then any two points can 
be connected by an admissible path [2]. Therefore, under this assumption, the cost 
Ct in (11.31) is well-defined for any T > and any points x,y on the manifold M. 

In this paper, we prove continuity of the optimal control cost Ct under some 
growth and convexity conditions on the Lagrangian L (see Theorem 13.21) . A simple 
useful corollary of the general continuity result is as follows: 

Theorem 1.2. Assume that the Lagrangian L and the vector fields X±, X n satisfy 
the following conditions: 

(1) C x \u\* + K x <L(x,u) <C 2 \u\ 2 + K 2 , 

(2) |^|<C7 3 M 2 ; 

(3) the Hessian of L in the u variable is positive definite, and 

(4) {Xi,...,X n } is ^-generating 

for some constants Ci, C 2 , C3, K\, K 2 > and some constant q > 1. Then the cost 
function (t,x,y) 1— > ct(x,y) defined in (Q73j) is continuous. 

As an application, we prove a version of the weak KAM theorem corresponding 
to the above optimal control cost c. More precisely, let H : T*M — > R be the 
Hamiltonian function defined by 

(1.4) H(x, a x ) = sup [a x (F(x, u)) - L(x, u)} 

ueu 

Note that the Hamiltonian H is, in general, neither fiberwise strictly convex nor 
coercive, which are basic assumptions on the classical weak KAM theory (see [9]). 

Theorem 1.3. (Weak KAM Theorem) If we make the same assumptions as in 
Theorem [Q| then there exists a unique constant h such that the Hamilton- J acobi 
equation U.l\) has a viscosity solution. 

The structure of this paper is as follows. In Section 2, we give a counter example 
showing that the 3-generating condition in Theorem 11.21 is essential. Section 3 and 
Section 4 are devoted to the proof of Theorem 11.21 and II. 3\ respectively. In Section 
5, we study a generalization of the Aubry-Mather problem to the present setting. 



2. Example 

Assume that M is two-dimensional and the control system has the form: 

±1 = Ui, x 2 = x\ + u 2 x\, 
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in some local coordinate chart. The family of vector fields X\(xi,x 2 ) = (1,0) and 
X 2 (xi,x 2 ) = is (k + regenerating but not /c-generating. In this section, we 

show that the cost function c\ corresponding to the Lagrangian L(x, u) = \{u\ + u^} 
is not continuous if k > 3. This shows that the 3-generating assumption in Theorem 
11.21 is essential. More precisely, 

Proposition 2.1. Assume that k > 3. Then the cost function c\ corresponding to 
the above control system and Lagrangian satisfies 

ci((0, w), (0, w)) = 0, ci((0, w), (0, z)) > K 

for some constant K > 0, all w, and all z < 0. In particular, the cost function c\ is 
not continuous. 

Proof. According to the result in [5], the cost function c\ is much better than con- 
tinuous (in fact semiconcave) at (x, y) if the points x and y are not connected by 
abnormal minimizers (see [1] or below for the definitions of normal and abnormal 
minimizers). Therefore, let us apply Pontryagin maximum principle and find can- 
didates for which the cost function c\ is not continuous. 
Let H£ be the Hamiltonian function defined by 

H^(x,p) = p(F(x, it)) + uL(x, u). 

By applying Pontryagin maximum principle (see, for instance, [I]), any minimizer 
(x(-),u(-)) of the minimization problem in (11.31) satisfies 

(2.5) *, = ^, ft = -fJ, fUo 

OPi OXi OUi 

for some curve p(-) and some constant v such that (u,p(t)) ^ 0. Moreover, v can be 
chosen to be either or —1. A minimizer (x(- ),«(•)) is abnormal if the corresponding 
v in the Pontryagin maximum principle is 0. It is normal if v = — 1. Note that a 
minimizer can both be normal and abnormal. 

In the present case, the Hamiltonian H£ is given by 

H v u {x,p) = pxiii + p 2 x\ + p 2 u 2 x\ + ^(ul + ul). 

For the abnormal case v = 0, (12. 5p becomes 

xi = Mi, x 2 = x\ + u 2 x\, p 2 = 0, pi = 0, p 2 x\ = 0. 

By Pontryagin maximum principle, p\ and p 2 cannot be equal to zero simultaneously 
since v = 0. It follows that x± = and £2 = ^2(0). The corresponding controls to all 
these paths are all given by the zero control u = 0. It follows that Ci((0, w), (0, w)) = 
for any w and these are candidates for discontinuities of the cost c\. 

Next, we show that Ci((0, w), (0,z)) > K for some constant K > and for all 
2 < w. For this, we consider the case v — — 1. In this case, the Hamiltonian is given 
by 

H u l ( x iP) =PiUi+p 2 x\+p 2 u 2 x k 1 - -(ul + uf). 
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It follows from ( 12.5ft that we have 

(2 g) H := H-\x,p) = -p\ + \xf V \ + xjp 2 , 

%i=Pi, P2 = 0, ui=Pi, u 2 = x\p 2 . 

If we assume that x(0) = (0, w) and x(l) = (0, z) with z < w, then it follows from 
( 12. 6ft that the cost ci((0,w), (0,2;)) for going from (0,w) to (0, z) is estimated by 

(2.7) Cl ((0,w),(0,z)) = l -j^p\ + xfpldt >\f Q 'P\dt. 

Since P2 is a constant of motion, we can fix p 2 and look at the phase portrait of 
the system 

xi=pi, pi = -kxf~ l pl - 2xxp 2 

(see Figure 1). The cost c((0, w), (0, z)) in (12.71) can be estimated from below by the 
area enclosed by the level set H = 0. More precisely, 

(2.8) c((0,«;),(0,z))> / p 1 {x 1 ,p 2 )dx 1 , 



where Pi(x,p 2 ) is defined implicitly by \p\ + \x^p2 + x\p 2 = and k = ^— 2t 2 is 

the positive zero of the function pi(xi,p 2 ). Note that p 2 < 0. Indeed, H(x(t),p(t)) 
is constant, we have H(x(t),p(t)) = if~ 1 (x(0),p(0)) > 0. It follows that p 2 (t) = 
p 2 (0) < 0. 

If we do a change of variable xi = kz, then we have 



(2.9) 



f K f K i 

/ Pi{xi,p 2 )dx 1 > / (-2x\p 2 - xf pi) 2 dxi 
Jo Jo 



22k-2 (_p 2 )2fe-2 f 







On the other hand, by Figure 1 and (12. 6p . we have 



i 

2fc-2 



(2.10) -f pjdt > - [ \pi\dt>- [ xidt > )- (-— } 

2 io 2 y 2 7{t|ii(t)>o} 2\ p 2 J 

If we combine (jO) . (I2TB1 . (I2T9D . and (12301 . then we get 

{/ 1 \ 2fc^2 ^3 

for some constant C > 0. 

It follows that the cost c((0,w), (0, z)) is bounded below by a positive constant 
independent of p 2 if fc > 3 and this finishes the proof of the result. □ 
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Figure 1 . Phase portrait of the example (level sets of the Hamilton- 
ian H) 



3. Continuity of Optimal Control Costs 

In this section, we will state and prove the general continuity result (Theorem 13. 2p 
mentioned in the introduction. To do this, let us introduce some notations. If X t is 
a, possibly time-dependent, vector field, then the corresponding flow tp t defined by 
tpo(x) = x and j^Lp t (x) = X t ((ft{x)) is denoted by 

(p t = exp / X s ds. 
Jo 

We define the endpoint map End^ o : L p ([0,T], U) — > M by 

Endl o (u(-)) = exp / F u{s) ds(x ), 
Jo 

where F u is the vector field defined by F u {x) = F(x, u) = X (x) + Y^=i UiXi(x). 

Let us first fixed a control u(-). The first goal is to show that the control system is 
locally controllable. It means that we can reach any point near the point End£ Q (u(-)) 
by adding a small control v(-) to the fixed one «(•). The first idea is to replace the 
control system (11.21) with drift Xq by one without drift. However, the control vector 
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fields Xi,...,X n will become time dependent in the new control system. This is 
accomplished in Lemma I'd. 11 Recall that if P : M — > M is a diffeomorphism and X 
is a vector field on M, then the pull back vector field P*X is the vector field defined 
by P*X = dP~ l (XoP). 

Lemma 3.1. Let g\ be the time- dependent vector field defined by 

g\ := ^exp F u ^ds\ X>. 

Then 



End l ( u (-) + v (-)) = exp / F u{t) dt o&Ep Y]vj(t)g$dt(xo). 

Jo Jo i=1 

Proof. Let Q t and R t be the flows exp f Q * F u ( s ) +V ( s )ds and exp J Q * F u ( s )ds, respectively. 
Let P t be the flow defined by Q t = Rt ° Pt- If we differentiate the above equation, 
then we get 

F u (t)+v{t) oQ t = Q t = R t oP t + dRt(P t ) = F u(t) o Q t + dR t {P t ). 

After simplifying the above equation, we get P t = dR~[ 1 {F v {t) o Q t ) = (R t )*F v ( t ) o P f 
and this completes the proof. □ 

Recall that we want to show local controllability by varying v(-). In Lemma [3.11 
we have decompose the endpoint map End Xo (u(-) + v(-)) into two parts. The first 
part exp J Q T F u ^dt is independent of the varying control v(-) and it is a diffeomor- 
phism. Therefore, it is enough to show local controllability for the second term 
exp J Q T Y^ii=i v i(t)9idt(xo) which is the endpoint map to a new control system 



(3.11) ± = 2^v i {t)gl 

i=i 

Note that this is a system with no drift but with time dependent control vector 
fields g\ as mentioned earlier. 

Before proceeding to the proof of local controllability of the system ( 13. lip , let us 
state the main result of this section which includes Theorem 11.21 as a corollary. 

Theorem 3.2. Assume that the Lagrangian L and the family of vector fields 

{<7*,...,<7*|te[0,r]} 

satisfy the following conditions: 

(1) dluF + ^KLfau) <C 2 \u\ p + K 2 , 

(2) |^|<C7 3 M 2 ; 

(3) the Hessian of L in the u variable is positive definite, and 

(4) {g\,...,tf n | t G [0,T]} is k-generating, Vtt(-), 

for some constants C±, C%, C3, K±, K 2 > and some constant q > 1. Suppose further 
that one of the followings is satisfied: 
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(1) k — 3 and p < 2, or 

(2) k>3,p< §5§. 

T/ien i/ie cost function (t,x,y) h- > c t (x,y) defined in U.3\) is continuous. 

Going back to the local controllability issue of the system (13 .lip , let us denote the 
endpoint map to the new system by $ T : L p ([0,T], U) x M — > M. More precisely, 

fT n 

$ T (v(-),x) := exp / V vAt)qUt(x 



exp / yZvi(t)gldt(i 
Jo i=i 



If the control vector fields in the above new system is time independent, then 
local controllability follows from the Chow-Rashevskii theorem (see for instance 
|13j). More precisely, we will need the following lemma for which the proof will be 
given for completeness. Recall that if X, Y are two vector fields, then the vector 
field adxY is defined by adxY = [X,Y]. 

Lemma 3.3. Let g±, ...,g n which are time-independent family of vector fields. Then 
there exists piecewise constant control w(-) for which w(t) has only one nonzero 
component for each t and such that 



(3.12) 



Z^exp^ e h>2wi(t)gj\ dt(x )) 

= f(x ) + e k (ad gi ...adg k _ 1 g k )f(x ) + o{e k ) 
as e — > for every smooth function f . 



Proof. Let P t 6 and Q\ be the flows corresponding to the control system ( 11.21) with 
controls ew p and ew®, respectively. More precisely, 



e w i ^ 9i J ds > Qt( x o) = exp J e \ 



w ?( s )9i ds. 



Moreover, assume that there are vector fields X and Y such that the flows P t e and 
Ql satisfy 

f(P t e (x )) = f(x ) + eXf(x ) + o(e), f(Ql(x )) = f(x ) + e k Yf(x ) + o(e k ) 

for all smooth functions /. 

Next, we define a control w which is the concatenation of the controls w p , W®, 
—w p , and — w®. 



w(t) 



f -w p {t) ifO<t<T 

-w Q (t-T) if T < t < 2T 

w p {t - 2T) if 2T < t < 3T 

w Q (t-3T) if3T<t<4T. 
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It follows that 



/ ^exp J e (\2wi(s)g- \ ds(x )^j = f{Q\ 



^oP^o(Q^)- 1 o(P^)- 1 (x )). 

Let h(ei,e 2 ) = f{Qx ° P? ° (Qt)~ 1 ° (-^t 1 ) -1 ^)) an d we want to consider the 
expansion of the function h(e, e) in the parameter e. Note that P® = is the 
identity transformation. It follows that the zeroth order term of the expansion of 
h(e, e) in e is f(xo). In fact, the following is true. 

(3.13) h(e u 0) = h(0,e 2 ) = f(x ). 



By definition of the flow Q e t , we have d\f{Qf) 



follows that dlh 



= for all i = 1, ...,k- 1. It 

e=0 



62 £2=0 



for each such i. Therefore, except the zeroth order term, 

any term of order less than k in the expansion of h vanishes. However, by (13.131) . 
the fc-th order vanishes as well. Therefore, we consider the (k + l)-th order term. 
Moreover, by the same argument, the only nontrivial (k + l)-th order term is given 

by d ei d^ 2 h . A computation shows the following 

£1=62=0 

«9 e fc+1 /i(e, e) = (* + l)9 ei 5* /i(ei, e 2 ) = (k + l)[Y, X]f(x ). 

£=0 ei=£2=0 

In conclusion, we have shown that 

/ ^xp^ e fX^i( s )^j ds(x ) \ = f(x ) + ^-[Y,X]f(x ) + o(e 

By rescaling time and multiplying the control w by a constant, we have a control 
w which satisfies 



0,4) f(*t£<(± 



uiiOOft I ds(x ) I =/(i )+e'=+ 1 [y,X]/(x )+o(f t + 1 ) 



Note that if the controls w p and w® are piecewise constant and have only one 
nonzero component for each time t, then so is w by construction. 

If we let the control w p and be the constant controls defined by wf (t) = 8^ 
and wf(t) = 5 it i 2 for each t, then (I3.14p shows that 
t / n \ 

/ ( exp / e ( ^iwi(s)^ rfs(xo) ) = f(x ) + e 2 ^, fi-iJ/Oo) + o(e 2 ). 



,i=i 



This proves the lemma for the case k = 2. The rest follows from induction using 

(HDD. □ 



The second idea is to take a control given by Lemma I3.3[ rescale it so that it is 
concentrated on a smaller and smaller time interval, and put the rescaled controls to 
the place where the vector fields g\, <?* are bracket generating. This way we obtain 
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local controllability as in Chow-Rashevskii theorem. Here we need the conditions 
on the numbers k and p to make sure that the rescaled controls stay small. This 
second idea will be achieved in Proposition 13.41 below. To do this, let us consider 
the curves t i— > gj(xo) contained in the tangent space T X0 M. Let X be an interval in 
[0, T] with the property that any subinterval X' contained in X satisfies 

span{a*(x ), (ar )|* e J '} = span{a*(x ), (sc )|t e X}. 

Proposition 3.4. Let r be a Lebesgue point of the control u(-) contained in the 
interval X and assume that either 

(1) k — 3 and p < 2, or 

(2) fc>3,p< §5§. 

Then there exists a, (3 > and a family of controls v e (-) which converges to in LP 
and such that 

/($V("))^o) = f(xo)+e k ^- a) [ T ad gJ ...ad gJ ( g r)f(x )ds + o(e k ^), 



as e — > 0, for any smooth function f . 

Proof. By Lemma [3 .3} there is a piecewise constant control w(-) for which w(t) has 
only one nonzero component for each t and such that 



o(e k ) 



(3 15) ; ' 

= f{x ) + e k (ad gIi ...ad gIk _ i gJ k )f( y x ) 

as e — ► 0. Note that r is fixed and g\ is a time independent vector field. 

Let = t < ti < ... < ti = T be & partition such that the restriction w\[ ti _ 1 ,t i ) of 
the control w(-) to the subinterval ij) is constant and there is only one nonzero 
component. We suppose that the fcj-th component of ^ | is nonzero and this 
nonzero component is equal to q. 

We need to create more freedom in our controls for later use (Lemma 13.51 to be 
precise). Let v(-) be a control of the form v(-) = w(-) +«(■) such that ctj\[ti,u +1 ) — 
if j 7^ ki and J ' 1+1 ai ti (s)ds = 0. It follows from (13.151) and J^ +1 ak i (s)ds = that 



(3.16) 



f(^$£e [itvi(t)gl\ dt(x ))J 



= f(x ) + e k (ad g r ...ad gJ g T ik )f{x Q ) + o(e k ). 

J- k — 1 

Next, we rescale the control v(-) as mentioned earlier. Let G SyV := Y^=i v i9i anc ^ 



let 

' e- a v((t-T)/e?) if t G (r, r + e 13 ) 
otherwise. 
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Then we have 

/ ^exp jf G s ^ {s) ds(x ) 
= f ( exp y G s ^ {s) ds(x ) 



By using the asymptotic expansion in [TJ section 2.4.4], the above equation be- 
comes 

fT 



(3.17) 



/ ^exp y C7 Sit „ (s) ds(x ) 
= /(^o) + ^2 £ l{P ~ a) G e fi Sl+TyV{si) ...G e e Si+TAsi) f(xo)ds l ...ds i + 

i=l ^0<si<...<Si<T 



k(J3-a)^ 



+ o(e 

as e — > 0. 

Let Jj be the term 

Jo<si<...<Si<r 

in the expansion (13.171) . 

Let us first deal with the term Ii(v(-)) = J G e s+TtV r s )f(xo)ds. For this, let 
9i +es = 9i + Zl' S - Let us recall that v(-) = w(-) + a(-) and <x,- = if j ^ fcj. 



h(v(-))= I ^v^g^fix^ds 

Jo i=i 

(3.18) =/ Y,v l (s)glf(x Q )ds+ y2(w t (s) + a t (s))Z € l ' s f(x )ds 

Jo i=1 Jo i=1 

^2vi(s)glf(x )ds + ^2 / (ci + a ki (s))Z t k ' i s f(x )ds. 

i=l i=l Jti-i 



The next lemma says that we can choose a to get rid of the last term of the above 
equation. 
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Lemma 3.5. There exists eo > and a e (-) in L 2 such that i a t k .(s)ds = and 

T n 

^2vi(s)glf(x )ds 
i=i 

for all < e < €q. Moreover, a e goes to in L 2 as e goes to 0. 

Proof of Lemma \3.5\ Recall that we need to satisfy the conditions 

(3.20) j ' al(s)ds = 0, f ' (a + al(s))Zl : s f(x )ds = 0. 

for all smooth functions / and for all %. Consider local coordinates around the point 
Xq and suppose that Z k ' s = (Z k s 1 , Z k s m ) in this local coordinates. Then the 
conditions in ( I3.20p is the same as that ct e k .{-) orthogonal to the constant functions 
and Q + a| (•) is orthogonal to Z ki in L 2 ([tj_i, tj\) for each i. Let Vi be the finite 
dimensional subspace of L 2 ([tj_i, U}) defined by 

V i ■= S P an {^fc-,l,"->^;',m}- 

A linear algebra argument shows that a € (-) which satisfy the conditions (I3.20p 
exist if Vf does not contain any nonzero constant function. Therefore, we assume 
that 

m 
3=1 

for some constants a±, a m , c, for all s in [tj_i, U], and for some i. We are going to 
show that c must be zero and this finishes the proof of the lemma. 

The above equation means that {Z k s x , Z^ ,s m ) is contained in the affine space 
{z G M m | YlT=i a i z i = c } ^ or almost all s in the interval Therefore, 
(-^Z k s 1 , ^^' s m ) is contained in the subspace {z G M. m \ YlT=i a i z i = 0} f° r eacn s 
in the interval [tj_i, tj\. Let us choose eo such that r + te is contained in the interval 
X for each t in [0,T] and for all e < eo- Then it follows from the definition of the 
interval J that (£ Zl' s (x ), £Z^ s (x )) is contained in {z G K m |E^i a ^i = °) 
for almost all s in [0, T]. Therefore, (Z^^(xo), Z k ^ m (xo)) is contained in the affine 

space {z G M m | X^Li = c ) f° r a ^ s i n [O'^l- However, (Z k ° l7 ^° m ) = 0, so 
c = and this finishes the proof of the lemma. □ 

For the rest of the proof, we write v(-) — w(-)+a £ (-) and suppress the e-dependence 
on v to avoid complicated notation. 

Lemma 3.6. 

r n 

h{v{-))= V v h (s 1 )...v ik (s ik )gl...glj(x )ds 1 ...ds k + o(l) 

Jo< Sl <...< Sk <T iu ik=l 

as e — > 0. 



(3.19) 



AM-) + c/(-)) 
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Proof of Lemma \3.6[ This follows immediately from the definition of G T:V . Indeed, 



Ge> 3 s 1 +T,v(s 1 )---GE> 3 s k +T,v(s k )f(. x o)dSl...ds k 



0<s 1 <...<s k <T 



„ n 

( 3 - 21 ) = / E ^(*i)..-^(**)^* 1 --^'V(xo)d»i...ds* 

J0< Sl <...<s k <T iu ik=1 
» n 

= V ^ 1 (s 1 )...^(s ifc )ft r i ...^y(x )rfsi...ds A; + o(l). 

■ / 0<-i<...<5 fc <T il>iii>ifc=1 

□ 



If we combine Lemma [331 and Lemma kT6l with (I3.17P and assume that 3/3 — 2a > 
k(j3 — a) > 0, then we have 



l-T 

f ( exp I G s>v e^ds(x ) 



\ Jo 

k 

f(x ) + E ^"^(O + + 0(e fc (^) 

i=l 

A; „ n 

,=1 J0<s 1 <...<s j <T i ■ x 



as e — ■> 0. 

By (13.161) . the above becomes 



/ ^exp jf G Sjt , E(s) rfs(a;o) 

= / ^exp^ ^~ a E^(*k T dn +o(e fc(/3 ~ a) ) 

= /(%) + eW-«\adl t ...ad giki g ik )f{x Q ) + (e fc ^)). 
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Finally, we need v e (-) converges to in LP . Indeed, by the definition of v e 
have 



we 



\v e (t)\ p dt 



t - T 



:s)\ p e^ds 



dt 



\w 



(s) + a e (s)\ p e^~ ap ds. 



Since w(-) is in L°° and a e (-) is in L 2 , v e (-) converges to in L p if (3 — ap > and 
V < 2. 

In conclusion, if we can choose a and j3 such that the following three conditions 
are satisfied, then the conclusion of the theorem holds. 

3/3 - la > k{/3 - a) > 0, (3 - ap > 0, p < 2. 

It is not hard to check that these inequalities are satisfied under the assumptions 
of the proposition. □ 

The local controllability of the control system follows using Proposition 13.41 and 
implicit function theorem as in the Chow-Rashevskii theorem. Finally, the continu- 
ity of the cost follows from the local controllability and standard arguments as in 
0- 

Proof of Theorem \3.2l Lower semi-continuity of the cost can be proved in the same 
way as in [5]. To prove upper semi- continuity, we let (xi,yi,ti), (£2,2/2,^2), ••• be a 
sequence of points which converges to (x,y,T) and lim^oo c ti (xi,yi) = r. We want 
to show that or(x, y) > r. 

Assume that this is not the case. Let u(-) and x(-) be a control and the tra- 
jectory associated to this control, respectively, such that x(0) = x, x(T) = y and 
J Q L(x(s),u(s))ds < r. Recall that the family of vector fields {g{, \t G [0,T]} 
is /c-generating. Therefore, we can find vector fields V\, Vu from the the set 



ad n t ...ad n 



1 < ij < n,0 < t < T,0 < / < fcj . 



which span the tangent space T X M. We also assume that Vi is defined by the Lie 
brackets of Ki vector fields of the form gj\ By perturbation, we can assume r, 7^ Tj 
for i 7^ j and that each Tj satisfies the condition in Theorem 13.41 Therefore, by 
Theorem 13. 4[ there is a family of control Wi^(-) such that 

/($ r K e (-),* )) = f(xo) + e K ^~ a) [ T V l f(x )ds + o(e^~^). 



Note that from the proof of Proposition [3]H we can assume that Wi M is supported 
in a small interval Ji around by taking (ei, e n ) small enough. Moreover, we can 
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assume that the intervals Jj are disjoint. We define the map f :Mx M. n x (0, oo) — 
M by 

(z, €!,..., 6 n , T) = $ T (Wi i£l/Kl( ^_ a) ) O <S> T {w 2 , e2 /K 2 (P-a)) ° ••• ° * T (^n,e„/ Kn (/3- Q ) ) (x) • 



Since 



e,=0 



^ = Vi, the map \l/ is of full rank at the point (x, 0, 0, T). It 



follows from implicit function theorem that there exists a map ift : f/i — > U 2 
from a neighborhood U\ of (x,y,T) to a neighborhood f/2 of (0, ...,0) such that 
i/)(zi, z 2 , t),t) = z 2 for all pairs (z±, z 2 , t) in the set U\. 
Let (e\, ■■■,e l n ) = ijj{xi,yi,ti) and let v l (-) be the control defined by v l (t) = w^ e i(t) 

and otherwise. v l (-) is well defined if (e\,...,e z n ) is close enough to (0, ...,0). Let 
x % {-) be a curve in M which satisfies ( II. 2p with control «*(■). We know that ?/(•) 
converges strongly in L p to and x l (-) converges uniformly to x(-). 
Assume, without loss of generality, that u(t) = for all t > T. Then 



(3.22) 



< 



L(x l (s),u(s) +v l (s))ds - / L(x(s),u(s))ds 

Jo 

L(x l (s),u(s) + v l (s))ds - / L(x(s),u(s) + v l (s))ds 

Jo 



+ 



+ 



+ 



L(x(s), u(s) + v l (s))ds — / L(x(s),u(s))ds 



+ 



max{L(x(s), u(s) + v i(s)), L(x(s) , u(s))}ds 



where 7j = min{T, tj}. 

Since L(x,u) < C 2 \u\ p + K 2 , ti converges to T, and the sequence u(-) + v l 
converges to u(-) in L p , we have 



(3.23) 



max{L(a;(s), u(s) + v 1 (s)), L(x(s), u(s))}ds 



T 



< / m&x{C 2 \u{s) + v l {s)\ p -K 2 ,C 2 \u{s)\ p - K 2 }ds 
Ju 

-> 



as 1 — > 00. 

Recall that |^| < C7 3 |u| 2 , where the norm is taken with respect to certain Rie- 
mannian metric. Let d be the corresponding Riemannian distance function. Then 



ANDREI AGRACHEV AND PAUL W.Y. LEE 



L(x l (s),u(s) + v l (s))ds - / L(x(s),u(s) + v\s))ds 
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we have 



(3 24) < J \L(x l (s),u(s)+v l (s))ds-L(x(s),u(s)+v l (s))\ds 

< sup d(x % (s), x(s)) / C 3 \u(s) + v l (s) I cis 

s Jo 

^ as i —> oo. 

By construction of the control w *(■), we know that the indicator function ^{t\ Vi (t)^o} 
converges to zero almost everywhere. It follows that 



(3.25) 



L(x(s), u(s) + v l (s))ds — / L(x(s),u(s))ds 

Jo 

rn 

< I I{ t \ v i(t)^o}{\L{x(s),u(s) + v l (s))\ + \L(x(s),u(s))\)ds 
Jo 

— > as i — > oo. 



Therefore, if we combine f l3T2"2"j) . (13^231) . (|33i]l . and (13^5) . then we have 



lim / L(x l (s),u(s) + v % (s))ds = / L(x(s),u(s))ds < r. 
Jo Jo 

On the other hand, 

/•ti 

lim / L(x l (s),u(s) + v l (s))ds > lim c ti (xi, i/i) = r. 



Therefore, this gives a contradiction and we finish the proof of upper semi- 
continuity of the function (t,x,y) i— > c t (x,y). □ 



4. Optimal Control and Weak KAM Theorem 

In this section, we give a proof of Theorem 11.31 using some ideas from [3] and [I] . 
More precisely, we will prove the following. 

Theorem 4.1. Assume that the function (t,x,y) h- > c t (x,y) defined by U.3\) is 
continuous and the manifold M is compact, then there exists a unique constant h 
such that the Hamilton- J acobi- Bellman equation U.l\) has a viscosity solution. 

We start the proof by introducing the Lax-Oleinik semigroup: 



(4.26) 



Stf(y) = inf \ct(x,y) + f(x)}. 
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Theorem 4.2. Assume that the function (t,x,y) i— > c t (x,y) defined by (CQJ) is 
continuous. Then, for each function f , the function (t,x) h- > S t f(x) is continuous 
on (0, oo) x M. Moreover, it is a viscosity solution to the Hamilton- J acobi- Bellman 
equation d t f + H (x, d x f) = on (0, oo) x M. 

Proof. Continuity of the function Stf follows immediately from that of c± and com- 
pactness of the manifold M. The fact that it is a viscosity solution follows as in 
0. □ 

The following theorem is a continuous version of [U Lemma 9] and the proof is 
similar. 

Theorem 4.3. Assume that the function (t,x,y) h- > ct(x,y) is continuous and the 
manifold M is compact. Then, for each a > 0, the family {c t \t > a} is equicontinu- 
ous. Moreover, there exists constants h and K such that 

\ct(x,y) - ht\ < K 

for allt> a and all x, y in M. 

Proof. The function (t,x,y) i— > c t (x,y) is uniformly continuous on [a,b] x M x M 
for some constants b > 2a > 0. So, given e > 0, there is a 5 > such that 

|c T (xi,yi) - c T (x 2 ,y 2 )\ < e/2 

whenever d(xi,x 2 ) < 5, d{y 1 ,y 2 ) < S and a < r < b. 

Assume that t > a. Since b > 2a, there is a partition = to < t\ < t 2 < ... < ti = t 
such that a < t i+ i — tj < b. Assume that c t (xi,yi) > c t (x 2 ,y 2 ) and let x 2 = 
z , Z\, zi = y 2 be points on the manifold M such that q(x 2 , y 2 ) = J2\=i c u( z i-i: z d- 
Then we have 

ct(xi,yi) - ct(x 2 ,y 2 ) 

< c tl _ to (xi,zi) - c tl - t0 {x2, Zi) +c tj _ tj _ 1 (^_i,?/i) - c tl -t l _ 1 {zi-i,y 2 ) 

< e 

whenever d(xi,x 2 ) < 5 and d(yi,y 2 ) < 5. It follows that {ct\t > a} is an equicon- 
tinuous family. 

Let M t = sup xy c t (x,y) and m t = mi Xty c t (x,y), where the supremum and the 
infimum are taken over all pairs of points of the manifold M. Let ti and t 2 be two 
positive numbers and let z be a point on the manifold M such that c tl+ t 2 (x,y) = 
c tl (x, z) + c t2 (z,y). It follows from this M tl+t2 < M tl + M t2 . Similarly, m t satisfies 
m t 1 +t 2 — m t 1 +rn-t 2 - It follows that the infimum of the function ^ is finite. Indeed, 
if the infimum of ^j- is — oo, then so is But note that -j^- > -j^- for all positive 
integer k. This gives a contradiction. It follows that M := inf t ^ is finite. Given 

e > 0, we find t such that < M + e. Every t > t can be decompose into 
t = kto + s, where t < s < 2t . It follows that 

„ , M t kM tn + M s M tn kt M s . , , , kt M s 

M < — - < = — — + — < (M + e)- — + — . 

t t to kto + s t kto + s t 
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By continuity of the cost c, we know that M s is bounded. It follows that from this 
and the above inequality that 

lim — - — M. 

t— >oo t 



Similarly, we also have 



m t 

hm — = m. 



t— too t 

Finally, it follows from equicontinuity of the family {ct\t > a} that M t — m t <C for 
some constant C and for all t > a. Therefore, h := M = m. □ 

Lemma 4.4. Assume that the function (t,x,y) i— »• c t (x,y) is continuous and the 
manifold M is compact. Let f be a bounded function, then the family S := {S t f — 
ht\t > a} is uniformly bounded and equicontinuous. 

Proof. According to Lemma 14.31 the family {c t \t > a} is equicontinuous. So, for 
each e > 0, there is a 5 > such that for all t > a 

\ct(xi,yi) - c t (x 2 ,y 2 )| < e/2 

whenever d(xi, x 2 ) < 5 and d(yi, y 2 ) < 5. 

By definition of Stf, we can find, for each e > 0, a point z t such that 

Stf(x 2 ) > c t {z t ,x 2 ) + f{z t ) - e/2. 

It follows that 

Stf{xi) - S t f{x 2 ) < c t (z t , xi) + f(z t ) - c t (z t , x 2 ) - f(z t ) + e/2 < e. 

Since the above equation holds for all e and all t > a, we conclude that the family 
S is equicontinuous. 

Fix a point x in M. For each e > 0, let z be a point in M such that 

ct(z,x) + f(z) > Stf(x) > c t (z,x) + f(z) - e. 
Therefore, by Theorem 14.31 we have 

K + sup{/(y)} > Stf(x) -ht>-K+ inf {/(?/)} - e 

yeM y^ M 

for some constant K > 0. We conclude from this that S is uniformly bounded. □ 
Define the function / by 

f(x) = inf[S t f(x)-ht]. 

t>a 

It follows from Lemma H~4l that / is bounded. The following theorem taken from [9] 
together with Theorem 14.21 and Lemma 14.41 finish the proof of the existence part of 
Theorem 14.11 We give a sketch of the proof here. 
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Theorem 4.5. Assume that there exists a constant h such that the family S := 
{Stf — ht\t > a} is uniformly bounded and equicontinuous, then S t f — ht converges 
uniformly to a function f. Moreover, it satisfies 

Sj-ht = f. 

Proof. By applying 5* to the definition of /, it is not hard to see that S t f — ht > f. 
Since St is order preserving, we can apply St again to this inequality to shows that 
t i— > Stf{x) — ht is increasing for each x in M. It follows from this and Lemma [4.41 
that S t f(x) — ht converges uniformly to a continuous function /. We apply once 
again S t to the definition of / and use the continuity of the semigroup St, we get 
Sj-kt = f. □ 

Finally, we finish the uniqueness of the constant /iasa corollary of Theorem 14.31 

Corollary 4.6. Assume that the function (t,x,y) >— > ct(x,y) is continuous and the 
manifold M is compact. Let h be as in Theorem \4-3\ and let f be a function which 
satisfies Stf — kt = f for some number k, then k = h. 

Proof. For each natural number n, let z n be points in M which satisfies 

c n (z n ,x) + f(z n ) > f(x) + kn = S n f(x) > c n {z n ,x) + f{z n ) - -. 

n 

Note that the function f is continuous and lim Sni2a£l = fa ft follows that if 

n—>oo 

we divide the above inequality by n and let n goes to infinity, we get k = h as 
claimed. □ 



5. Optimal Transportation and Weak KAM Theorem 

Let fi and v be two Borel probability measures. Consider the cost function defined 
in ( I1.3P and the following Monge-Kantorovich problem of optimal transportation: 

(5.27) C T (ji, u) = inf / c T (x, y)dU(x, y) 

n J MxM 

where the infimum is taken over all measures on M x M with marginals fi and v. 
That is, if 7Ti, 7r 2 : M x M — > M are the projections onto the first and second entries, 
then 7r 1:( ,n = /j, and 712*11 = v. 

The above problem (I5.27P admits a dual version given by 

(5.28) 1t{h, v) = sup / g{x)dv{x) — \ f(x)fi(x), 

Jm Jm 

where the supremum is taken over all pairs of functions (/, g) which satisfy g(y) — 
f(x) < c T (x,y). 

The following theorem is the well known result in [TT] . See also [141 [T5] . 
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Theorem 5.1. Assume that the function ct is continuous, then the infimum in 
( 5.27\ ) and the supremum in 115. 28\) is achieved. Moreover, for any optimal measure 
II of \5.21ty and any pair of functions (f,g) that maximizes 115. 28\) , we have that IT 
is concentrated on the set {(x,y) G M x M\g(y) — f(x) = cx(x,y)} and CV(/i, v) = 
lr(ji, v). 

Note that if (f,g) maximizes (15.281) . then so is (/, St/)- We define 

(5.29) a T :=inf^L7 T (/i,/i), 

where the infimum is taken over all Borel probability measures on M. 
The following lemma can be proved in same way as in [21 Lemma 33]. 

Lemma 5.2. There exists a measure \i which achieves the infimum in K5. 29\) . 

The next theorem is a generalization of a result [3] which gives another charac- 
terization of the number h in Theorem 11.31 

Theorem 5.3. Under the assumptions in Theorem \1.3[ we have ax = h for each 
T > 0. 

Following 0] , we call measures II on the space MxM generalized Mather measure 
if 71"!* IT = 7r 2 *II and 

— / c T (x,y)dU(x,y) = h. 

1 J MxM 

The following corollary describes the support of the generalized Mather measures. 

Corollary 5.4. Suppose that we make the same assumptions as in Theorem \ 1 . 3\ and 
let g be a function which satisfies S t g = g + ht. If U is a generalized Mather measure 
which satisfies 

— I c T (x,y)dH(x,y) = h, 

1 JMxM 

then the support of U is contained in the set 

{{x,y)\c T {x,y) = g(y) -g(x) + hT}. 
Proof. Let \x = ttuII = 7T2*I1. Then 



C T (n, n) = hT= \ S T gdfi - / gdfi < J t (a*, aO = Ct(a*» ft). 
Jm Jm 

It follows that the support of II is contained in 

{{x,y)\cr(x,y) = S T g(y) - g(x)}. 

□ 
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Proof of Theorem \5.3[ Let g be a function which satisfies Stg = g + ht and let /i be 
a minimizer corresponding to the minimization problem of ax in (15.291) . It follows 
from Theorem 15.11 that 



(5.30) Ta T = C T (/i,/i) = T t (h,ij) > / S T gdji - \ gd/i = hT 

Jm Jm 

for all T > 0. 

For the proof of the following lemma, we follow closely [U Lemma 7] . 

Lemma 5.5. Let v\ and v 2 be two Borel probability measures and let < s < T , 
then there exists a Borel probability measure v such that 

C T {vi, v 2 ) = C s (vi, v) + C T _ s (i/, v 2 ). 

Proof. By Theorem 15 .11 we can find measures LTi, II2 on MxM such that C s (ui, v) = 
LixM c ^ x ^y) dIl ^ x ^y) and C T - S (v,v 2 ) = J MxM c T - s (x,y)dU. 2 (x,y). By disintegra- 
tion of measures, there are measures fi\ and /xf such that dlli(x,y) = dfi\(x)du(y) 
and dU 2 (x, y) = d/i% (y)dv(x). Let /i be the measure defined by 



f(x,y)dll(x,y) = / f(x,y)d{i z 1 (x)d{i z 2 (y)dv(z). 

MxM JMxMxM 

It is not hard to check that the marginals of II are v\ and v 2 . Therefore, we get 
C T (z/i, v 2 ) < I c T (x,y)dH(x,y) 

J MxM 

< / c 8 {x,z) +c T _ s {z,y)dfi z 1 {x)dfi z 2 {y)du(z) 

Jm xm xm 

= / c s (x,z)dll 1 (x,z) + / c T _ s {z,y)dIl 2 (z,y) 

J MxM J MxM 

= Csiyx, v) + C T - S {v, v-i). 

Let V be the set of pairs of Borel probability measures (i>i, u 2 ) which satisfies the 
conclusion of the lemma. It is not hard to see that (S x , S y ) is contained in V, where 
S x is the Dirac mass at x. Indeed, let x(-) : [0, T] — >• M be an admissible path which 
satisfy x(0) = x, x(T) = y and achieve the infimum in (11. 3p . Then, 

^ C S (5 X , 5 x[s) ) + C T _ S (5 X ( S ), S y ) = c s (x, x(s)) + c T _ s (x(s), y) 

= c T (x,y) = C T (S x ,5 y ). 

To finish the proof, it remains to notice that the set V is convex and weak-* 
closed. Therefore, the result follows from approximation by delta masses. □ 

Now let v be a measure which satisfies Cnt{v, v) = cynt- It follows from Lemma 
15.51 that there exists Borel probability measures v = /i , /xi, /i^ = v such that 

N 

C NT (u, v) = y^Cr^-!,^). 

i=i 
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Since Xt is convex and so is Ct- Therefore, 

C NT (v, v) > —C T (fi,fi) >a T >h, 



NT ~ T 



where ji=jj Ei=i V*- 

Finally, it follows from Theorem 14.31 that lim ■^Cjvt( z/ ) ^) = h. This finishes 

the proof. □ 
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